Transport improvements from 2024-W49 #259

khaeru · 2024-12-03T15:39:32Z

Items for SSP 2024/ScenarioMIP.
- Update base scenario URLs from e.g. ixmp://ixmp-dev/SSP_dev_SSP1_v0.1_Blv0.18/baseline_prep_lu_bkp_solved_materials to e.g. ixmp://ixmp-dev/SSP_SSP1_v1.1/baseline_DEFAULT_step_13.
- Constrain t=conm_ar (a P AIR technology) to a low/zero share in 2020. This technology was (a) taking a large share in y=2020 without constraints and (b) because not present at all in IEA EWEB for some regions, was being scaled with 1.0, so passing through as a large value.
- Use a smaller growth_activity_up value for (t=ICAm_ptrp, n=R12_RCPA). The technology constraint as of 2020 is effective, but this tech/region grows too rapidly to 2025 and beyond. (per discussion with @r-aneeque 2024-12-04.)
- Adjust efficiencies of LDV PHEV technologies using duty cycle data.
Items for aviation emissions postprocessing:
- Handle missing countries in IEA EWEB data #230

This branch is currently triggering the message_data transport runs using iiasa/message_data#582, which limits to a subset of all scenarios.

How to review

TBA

PR checklist

Continuous integration checks all ✅
Add or expand tests; coverage checks both ✅
Add, expand, or update documentation.
Update doc/whatsnew.

codecov · 2024-12-03T15:50:01Z

Codecov Report

Attention: Patch coverage is 53.20334% with 168 lines in your changes missing coverage. Please review.

Project coverage is 65.0%. Comparing base (d55fbf9) to head (2a9f63c).

Files with missing lines	Patch %	Lines
message_ix_models/testing/check.py	35.7%	122 Missing ⚠️
message_ix_models/model/transport/freight.py	0.0%	16 Missing ⚠️
message_ix_models/model/transport/non_ldv.py	18.1%	9 Missing ⚠️
...sage_ix_models/tests/model/transport/test_build.py	60.0%	8 Missing ⚠️
message_ix_models/util/genno.py	36.3%	7 Missing ⚠️
message_ix_models/tools/iea/web.py	94.2%	2 Missing ⚠️
message_ix_models/model/transport/build.py	50.0%	1 Missing ⚠️
message_ix_models/model/transport/config.py	96.7%	1 Missing ⚠️
message_ix_models/model/transport/ldv.py	0.0%	1 Missing ⚠️
message_ix_models/util/sdmx.py	95.8%	1 Missing ⚠️

❗ There is a different number of reports uploaded between BASE (d55fbf9) and HEAD (2a9f63c). Click for more details.

HEAD has 10 uploads less than BASE

Flag BASE (d55fbf9) HEAD (2a9f63c)

22 12

Additional details and impacted files

@@           Coverage Diff            @@
##            main    #259      +/-   ##
========================================
- Coverage   77.6%   65.0%   -12.6%     
========================================
  Files        211     213       +2     
  Lines      16079   16335     +256     
========================================
- Hits       12481   10623    -1858     
- Misses      3598    5712    +2114

Files with missing lines	Coverage Δ
message_ix_models/model/transport/demand.py	`40.8% <ø> (-57.8%)`	⬇️
message_ix_models/model/transport/files.py	`88.0% <ø> (-10.6%)`	⬇️
message_ix_models/model/transport/structure.py	`91.2% <100.0%> (+0.8%)`	⬆️
message_ix_models/model/transport/testing.py	`75.0% <ø> (-25.0%)`	⬇️
message_ix_models/model/transport/workflow.py	`79.6% <100.0%> (-1.9%)`	⬇️
message_ix_models/testing/__init__.py	`76.3% <100.0%> (-17.4%)`	⬇️
...age_ix_models/tests/model/transport/test_demand.py	`58.4% <ø> (-32.5%)`	⬇️
...age_ix_models/tests/model/transport/test_report.py	`67.6% <ø> (-24.0%)`	⬇️
message_ix_models/tests/tools/iea/test_web.py	`100.0% <100.0%> (ø)`
message_ix_models/model/transport/build.py	`17.4% <50.0%> (-78.0%)`	⬇️
... and 9 more

... and 37 files with indirect coverage changes

- Only write the file if it is different from a refreshed codelist. - Use ScenarioCodeAnnotations class. - Add Config.use_scenario_code() - Update base scenario URLs.

Replace with annotations in CL_TRANSPORT_SCENARIO.

This commodity is not used in IEA EWEB 2024 data for 2019, so we constrain its activity to a low, non-zero level.

Mirror calculation flow in .ldv for same parameter.

"output" for most technologies is dimensionless because ACT and the output commodities have the same units.

- Use .testing.check features to simplify list of assertions/debug output. - Add the same set of cases as .test_demand.test_exo.

Include all assertions in .test_build.CHECKS

…instead of "input" parameter values.

Use (regions=R12, years=B) to match 'main' model version.

Resolve infeasibilities with updated base model.

purely based on very high 2025 demand numbers that are unrealistic

purely based on very high 2025 demand numbers that are unrealistic. Aviation adjustment based on scale-1

based on scale-1 and high 2025 values

based on scale 1 and 2025 values (AIR is too high and demand is too high as well)

Remove values for: - R12_AFR, ROAD, ethanol - R12_FSU, ROAD, electr - R12_LAM, ROAD, gas - R12_MEA, RAIL, lightoil - R12_PAO, ROAD, gas - R12_RCPA, RAIL, lightoil - R12_RCPA, ROAD, gas - Add values for: - R12_AFR, RAIL, lightoil - R12_CHN, ROAD, electr - R12_NAM, RAIL, lightoil - R12_PAO, ROAD, ethanol - R12_RCPA, ROAD, ethanol - R12_SAS, ROAD, ethanol - Adjust all others

- Store context.core.local_data on pytestconfig for reference. - Add user_local_data fixture for development.

khaeru added transport MESSAGEix-Transport variant or transport in the base model p:SSP-2024 2024 SSP updates and ScenarioMIP labels Dec 3, 2024

khaeru self-assigned this Dec 3, 2024

khaeru force-pushed the transport/2024-W49 branch 6 times, most recently from e0e1fb6 to 8967c53 Compare December 5, 2024 13:40

khaeru added 20 commits December 17, 2024 11:03

TEMPORARY Run 'transport' CI workflow on PR branch

f5893f3

TEMPORARY Use branch for iiasa/message_data#582

e4cbf8c

Add .util.sdmx.AnnotationsMixIn

f47b1cf

Add .transport.config.refresh_cl_scenario()

234a26f

- Only write the file if it is different from a refreshed codelist. - Use ScenarioCodeAnnotations class. - Add Config.use_scenario_code() - Update base scenario URLs.

Remove data/transport/base-scenario-url.json

2b060da

Replace with annotations in CL_TRANSPORT_SCENARIO.

Simplify .transport.workflow via .use_scenario_code()

6fcdc13

Limit t=conm_ar in transport/R12/act-non_ldv.csv

38abb49

This commodity is not used in IEA EWEB 2024 data for 2019, so we constrain its activity to a low, non-zero level.

Include computed demand for transport F RAIL

9f53ca5

Generate output to "transport F RAIL vehicle"

2b54b5a

Mirror calculation flow in .ldv for same parameter.

Add "F" as a parent of freight modes; units

cab69a1

Copy "output" anno from F {RAIL,ROAD} to children

0d662a3

Silence warnings re: units in .transport.ldv

c908132

"output" for most technologies is dimensionless because ACT and the output commodities have the same units.

Add .util.genno.insert

9777fa3

Exclude "build" dir from mypy type checking

026f34d

Improve type hints in {,.transport}.testing

97fb43a

Add .testing.check

60bc5d9

Improve .transport.test_build.test_debug

4e8bd01

- Use .testing.check features to simplify list of assertions/debug output. - Add the same set of cases as .test_demand.test_exo.

Add "N_node" test fixture

85d2c5b

Drop .test_demand.test_exo

abb4505

Include all assertions in .test_build.CHECKS

Align keys in .transport.{freight,non_ldv} with .ldv

26b00ae

khaeru and others added 14 commits December 17, 2024 11:03

Allow no graphviz in .transport.build.get_computer()

ffea232

Derive freight cap. factor/lifetime from "output"

fabafae

…instead of "input" parameter values.

Add CPLEX time limit to .transport.workflow

7327f04

Reference .transport.files key in .non_ldv

757cd23

Adjust parametrization of test_report_base_solved

6fa1138

Use (regions=R12, years=B) to match 'main' model version.

Bump initial_{activity,new_capacity}_up values

7555730

Resolve infeasibilities with updated base model.

Set all minimum_activity values to 0.01

2bcf960

Reduce R12_WEU

d832004

adjust R12_WEU modes shares to match 2019 based estimates

2624594

Reduce R12_MEA

7fa070b

purely based on very high 2025 demand numbers that are unrealistic

Update R12_MEA mode share

3fffd77

purely based on very high 2025 demand numbers that are unrealistic. Aviation adjustment based on scale-1

R12_MEA freight-mode-share adjustment

e6ec62c

based on scale-1 and high 2025 values

R12_SAS mode share change

847affd

based on scale 1 and 2025 values (AIR is too high and demand is too high as well)

khaeru force-pushed the transport/2024-W49 branch from 3c9192d to b1f411e Compare December 17, 2024 10:03

khaeru added 4 commits December 17, 2024 17:16

Add transform="B" option for IEA EWEB data (#230)

d00c30e

Test transform="B" option for .iea.web

6243711

- Store context.core.local_data on pytestconfig for reference. - Add user_local_data fixture for development.

Edit IEA EWEB documentation

dbd1cad

Add --jobs=auto to default Sphinx options

2a9f63c

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Transport improvements from 2024-W49 #259

Transport improvements from 2024-W49 #259

khaeru commented Dec 3, 2024 •

edited

Loading

codecov bot commented Dec 3, 2024 •

edited

Loading

Transport improvements from 2024-W49 #259

Are you sure you want to change the base?

Transport improvements from 2024-W49 #259

Conversation

khaeru commented Dec 3, 2024 • edited Loading

How to review

PR checklist

codecov bot commented Dec 3, 2024 • edited Loading

Codecov Report

khaeru commented Dec 3, 2024 •

edited

Loading

codecov bot commented Dec 3, 2024 •

edited

Loading